Normal Models for Response Times on Test Items (CT 04-08) (PDF)

Two versions of a normal and lognormal model for the response times of examinees on test items are investigated. The models have a parameter structure analogous to the 2PL response models in IRT, with a parameter for the speed of each person as well as parameters for the time intensity and discriminating power of each item. It is shown how the parameters can be estimated by a Markov chain Monte Carlo (MCMC) method (Gibbs sampler). The method was used to analyze response times for the adaptive version of a test from the Armed Services Vocational Aptitude Battery (ASVAB). We tested the validity of the models using posterior predictive checks on the response times. The lognormal model showed an excellent fit to the data, whereas the normal model appeared unable to allow for a characteristic skewness of the response time distributions. The addition of an equality constraint on the item discrimination parameters did not lead to appreciable loss of fit. The potential use of the models for improving the daily practice of testing is indicated. Introduction It has long been known that response times on test items are an important source of information on the person's behavior, but we had to wait for the advent of computer-based testing to make the recording of response times a routine part of test administration. Now that testing is widely computerized, the question of how to model response times has become urgent. The current literature on test theory shows two fundamentally different approaches to modeling response times. One approach is to model these times in the framework of an item-response theory (IRT) model for the response variables for the same items. Examples of this approach are found in Thissen (1983), Roskam (1997), van Breukelen (1989), and Verhelst, Verstraalen, and Jansen (1999). In the other approach, response times are modeled independently of the response variables for the items. Examples of this approach are found in Maris (1993), Scheiblechner (1979), Schnipke and Scrams (1999), van der Linden, Scrams, and Schnipke (1999), and van der Linden and van Krimpen-Stoop (2003). A review of both types of models is offered in Schnipke and Scrams (2002). Typical of IRT models is that they describe the distribution of response variables using person and item parameters. Let Uij be the response variable for person j and item i, with Uij = 1 denoting a correct and Uij = 0 an incorrect response. One of the mainstream IRT models is the three-parameter logistic (3PL) model: Pr{Uij = 1} = pi(θ; ai, bi, ci) ci + (1 – ci)Ψ(θ), (1)